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ABSTRACT 

This report presents evaluation findings regarding the 
implementation and effectiveness of the Academics 2000, Cycle VIII grant in 
the Austin Independent School District (AISD) , Texas. The purpose of 
Academics 2000 was to raise the level of academic achievement of all Texas 
students by ensuring that each child achieves early mastery of the foundation 
subjects of reading, English language arts, mathematics, social studies, and 
science. AISD has funded programs for K-3 students through Academics 2000, 
beginning with Cycle V in 1998-1999. The focus of Cycle VIII efforts was 
third grade staff development and the improvement of student reading skills 
in science and social studies. To this end, 27 of the highest poverty schools 
in the school district received texts in the target areas and teacher 
professional development. The school year 2001-2002 was the final year for 
the grant. The report contains a brief description of the grant and its 
implementation in AISD, including the staff hired, the materials funded, the 
professional development provided, and the classroom activities performed 
through the grant. The report contains information on grant results related 
to changes in teaching practice, parent involvement, and student achievement. 
It also contains information on obstacles that may have decreased the 
effectiveness of grant activities. Recommendations are presented to continue 
some of the activities implemented through this grant. (SLD) 
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The purpose of this report is to present evaluation findings regarding the implementation 
and the effectiveness of the Academics 2000, Cycle VIII grant in AISD. 2001-02 was the final 
year for the grant, but the information in this report may be useful to district decision makers in 
identifying needs to be met through other means. 

This report provides a brief description of the grant and its implementation in AISD, 
including the staff hired, the materials funded, the professional development provided and the 
classroom activities performed through the grant. The report contains information on grant 
results related to changes in teaching practice, parent involvement, and student achievement. It 
also includes information on obstacles that may have decreased the effectiveness of grant 
activities. Finally, recommendations are presented for continuing some activities and for 
improving those activities. 

Academics 2000 funding was provided through the Goals 2000: Educate America Act for 
planning and implementation of initiatives to improve reading, including intensive and sustained 
professional development in research-based instructional strategies and methodologies. The 
purpose of Academics 2000 was to raise the level of academic achievement of all Texas students 
by ensuring that each child achieves early mastery of the foundation subjects of reading, English 
language arts, mathematics, social studies, and science. 

AISD has funded programs for K-3 students through Academics 2000, beginning with 
Cycle V in 1998-99. With Cycle VIII, district staff decided to focus efforts on third-grade 
classes, and to provide staff development and classroom support to improve students’ reading of 
expository texts in the areas of science and social studies. To this end 27 of the highest poverty 
schools in the district were provided with appropriately leveled texts in the areas of science and 
social studies, and selected teachers at those campuses were given training in teaching strategies, 
including Guided Reading, Balanced Literacy, the Principles of Learning, and the use of 
assessments to guide instruction. 

District staff informed principals at qualifying schools that they were eligible for the 
program. Campuses at which at least half of the third-grade teachers agreed to participate were 
included in the program. After receiving ti-aining through the grant, teachers were expected to 
provide mentoring and support to other teachers on their campuses. 
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Role of Reading coaches 

Seven reading coaches were hired to give training and to support teachers in using the 
targeted skills in the classroom. Coaches had received training in literacy, including Balanced 
Literacy and Literacy Backbone, and had taught K-3 for at least 5 years. Beginning in July and 
continuing throughout the school year, the coaches attended training in content reading and 
Content-Focused Coaching provided through a contract with the Institute for Learning (IFL). 
Initial training took place during the summer of 2001. All seven coaches responded to a survey 
on the usefulness of the training. Five coaches reported that it was very useful and two reported 
that it was somewhat useful. The coaches suggested that training would be improved if more 
time were available for discussion and review, and if the presenters offered more hands-on 
exercises and feedback. They also suggested that future sessions not be held during the summer. 

Training in Content-Focused Coaching continued over the course of the year. In a survey 
in May, all seven coaches reported that the training was somewhat useful. All the coaches 
suggested that more feedback would have been helpful; five wished for additional support, 
including one-on-one coaching from the IFL presenters; and three reported dissatisfaction with 
the unavailability of IFL presentation materials, including videos and copies of overheads. 
Professional development for teachers 

The coaches presented a curriculum of 9 professional development sessions with a 
combined attendance of 645 teachers and 17 administrators. Material covered in these sessions 
included components of Balanced Literacy, Guided Reading, the Principles of Learning, and use 
of assessments to drive instruction. In a survey conducted in May, all seven coaches reported 
that the professional development was very effective. The most commonly reported challenge 
was planning lessons that addressed the teachers’ various levels of experience with Balanced 
Literacy. Of 47 teachers who responded to a survey in May, 2002, 79% reported that the 
professional development was “very useful” and 21% reported that it was “somewhat useful.” 
Suggestions for improving the usefulness of the program included offering more one-on-one 
work with the coaches and offering professional development at different levels of experience. 

Coaching 

Coaches worked with individual teachers, helping them to plan and implement effective 
lessons. Teachers met with coaches before lessons to plan strategies and afterwards to evaluate 
the implementation and to plan future lessons. In a survey at the end of the year, six coaches 
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reported that coaching was “very effective,” and one reported that it was “somewhat effective.” 

In a survey for teachers, 67% of teachers responding reported that coaching was “very useful,” 
31% reported that it was “somewhat useful,” and 2% reported that it was “not very useful.” 

During the lessons, coaches observed, modeled strategies and gave assistance as 
necessary. Surveyed about modeling, six coaches reported that it was “very effective,” and one 
coach reported that it was “somewhat effective.” Among teachers, 64% of teachers responding 
reported that it was “very useful,” 32% reported that it was “somewhat useful,” and 4% reported 
that it was “not very useful.” Surveyed about classroom support or coteaching, five coaches 
reported that it was “very effective,” and two coaches reported that it was “somewhat effective.” 
Of teachers responding, 66% reported that it was “very useful,” 32% reported that it was 
“somewhat useful,” and 2% reported that it was “not very useful.” 

Family Participation 

Each campus in the district held at least two family literacy events during the year. 
According to attendance sheets from each campus, 1,017 parents attended these events. Average 
attendance at individual events was 23 parents. Observed events comprised a number of booths, 
organized by grade level and staffed by teachers and volunteer tutors, each with a game or 
activity that supported age-appropriate reading skills. Most literacy events followed this pattern, 
but a few campuses held different events, including a “readers’ tea” and a “writing night.” 
Collaborations 

AISD staff coordinated efforts with staff from several other organizations. The Austin 
Learning Academy offered English as a Second Language classes for family members of 
students. The 35 1 participating parents attended a total of 12,404 hours of classes. KLRU public 
television station sponsored Mother Goose, a 2-hour literacy activity for students and their 
families, which was attended by 49 parents. The Austin Public Library headed a drive to help 
students and parents get library cards. Education Service Center Region XIII helped to support 
professional development and participated in the broad-based planning panel. IFL provided 
training in Content Focused Coaching to reading coaches. Pacific Learning provided materials 
and training for Guided Reading. Program staff would like all of these collaborations to continue 
next year with other funding. AISD will contract with Institute for Learning for training in 
Content-focused Coaching next year. The relationship between AISD and Austin Public 
Libraries will continue through the READ for Texas grant. 
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Classroom prac i ice 

Over the course of the year, the evaluator observed 28 classes (14 classes in December 
and 14 classes in April) to evaluate the extent to which training was being implemented in 
classroom practice. A second evaluator observed nine of the lessons. Agreement between the 
observers was 88%, After each observation, the evaluators compared notes and reached 
consensus. Lessons were to cover language arts, and include strategies taught through the grant 
progi'am. The evaluator rated each class on implementation of program strategies, including 
Balanced Literacy, Guided Reading and the Principles of Learning that are cunxntly stressed in 
the district: Accountable Talk, Clear Expectations, and Academic Rigor (Table 1, below,) 



Table 1: Number of Classrooms Exhibiting Program Activities during Obseiwations, 



Observed activity 


December 

14i 


April 

(u=^|4) 


Number 
show iiig 
activity 


Percent 

sliowine 

aettvityss 


Guided reading 


10 


13 


23 


82 


Connections with science and social studies 


8 


10 


18 


64 


Texts at different levels of difficulty 


13 


13 


26 


93 


Learning Centers 


12 


13 


25 


89 


Students can explain what they are doing 


11 


1 1 


22 


79 


Students can judge work using criteria 


2 


3 


5 


18 



Most of the classes observed were implementing program strategies to some degree, Ot 
28 obseiwed classes, 23 did Guided Reading lessons. Of those, 18 were observed using 
expository texts with content in science or social studies. Also, 25 classes used a centers format. 



Most of the classrooms had criteria charts posted, but many of those charts were very 
general and not related to specific cuiTcnt assignments. There was little evidence that the 
students had been involved in explicating the criteria for their assignments. Many ot the criteria 
charts were prepackaged. 

Only two classrooms in December and five classrooms in April had rubrics posted, and 
there seemed to be some confusion about what rubrics were. Four classrooms had charts that 
were called rubrics but did not meet the definition given by the Institute for Learning, Students 
were able to describe what they were working on, but only five classes had one or two students 
who were able to tell the evaluator why they were working on it or to judge their own work 
based on explicit criteria. 
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As part of the April observation, the evaluator used a locally developed protocol for 
evaluating the Principles of Learning. The levels of accomplishment on the protocol for each 
Principle are, ‘absence,’ ‘initiation,’ ‘beginning,’ ‘accomplished,’ and ‘exemplary.’ Findings 
were compared to a districtwide pilot of the observation protocol in which 23 elementary classes 
were observed. 

The vast majority of classes at participating schools were in the lower stages of 
implementation with very few classes at the Accomplished or Exemplary stages (Figures 1-3). It 
should be noted, though, that although Academics 2000 schools lag behind the district, the 
pattern of implementation of is very similar to that of the district as a whole, especially in the 
areas of Accountable Talk and Clear Expectations. Implementation of the Principles of Learning 
in AISD is reported more fully in Principles of Learning, 2001-02 Evaluation Report (District 
Report 01.16). 

Figure 1 ; Classroom Observation Ratings for Implementation of Accountable Talk in the 
Academics 2000 Evaluation and in the Pilot for the Observation Protocol 
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Figure 2: Classroom Observation Ratings for Implementation of Clear Expectations in the 
Academics 2000 Evaluation and in the Pilot for the Observation Protocol 
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Figure 3: Classroom Observation Ratings for Implementation of Academic Rigor in the 
Academics 2000 Evaluation and in the Pilot for the Observation Protocol 
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Because these were single observations, they represent snapshots of classes. External 
factors, such as the TAAS, may have affected the ratings for some classes. 

Student achievement 

Classes that participated in the Academics 2000 grant had an average passing rate of 75% 
on the TAAS reading test. Non-participating classes at those campuses had a TAAS passing rate 
of 74%. The district passing rate on the reading TAAS for 2001-02 was 85%. Of the 27 schools 
in the grant, 17 had higher passing rates in 2001-02 than in 2000-01, but, overall, there was not a 
strong increase in TAAS performance that can be attributed to the Academics 2000 program. 
Challenges 

Students in Academics 2000 classes took the Flynt-Cooter reading test three times during 
the school year. Teachers used the results to gauge progress and tailor instruction to student 
needs. Those data are not available for evaluation because of confused communication in the 
district of who was responsible for the entry and oversight of information for the district 
database. That problem is being addressed by staff in central administration. 

Participating teachers were surveyed in April regarding challenges in implementing 
program strategies. The challenge most commonly reported by teachers was finding time to 
implement the strategies they were learning. Of 40 teachers responding, 17 (43%) reported that 
they did not have enough time to implement strategies. Another seven (18%) reported feeling 
overwhelmed by the information. Additionally, eight teachers (20%) reported that they did not 
have the materials they needed. In terms of the strategies themselves, four teachers (10%) 
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reported that they had trouble teaching centers, or didn’t believe in them and four (10%) reported 
having problems with classroom management, especially for students working independently. 
TAAS pressure was reported as an obstacle by two teachers (5%), and three teachers (8%) had 
difficulty going to off-campus training sessions. 

The challenge most commonly reported by reading coaches was variability in teachers’ 
levels of experience and knowledge of Balanced Literacy. Coaches also reported lack of 
feedback from supervisors as a challenge. 

The program manager reported that the biggest challenge this year was a shortage of 
funding. Because not all funding was spent in 2000-01, TEA reduced funding for 2001-02 by 
20%. According to an interview with the program manager, spending grant money was made 
difficult by purchasing problems, which sometimes made the order process take months, and by 
accounting problems at central office that made it difficult to tell how much money was 
available. In order to provide all promised services, program staff reduced the number of 
coaches to seven from a planned 12, and limited the support materials sent to campuses. 
Recommendations 

Teachers and coaches reported that the training received through the grant was useful, but 
it did not translate into improved TAAS scores. The most frequently cited obstacle listed by 
teachers that could account for this, is that teachers did not have enough time to implement the 
strategies they were learning. It is recommended that the district continue to support training 
efforts by helping teachers to structure their days such that they could find time to absorb and use 
the strategies taught by the program. 

The current budget includes a plan to continue the contract with IFL to provide 
professional development in Content Focused Coaching. Also, materials and some trained staff 
are already at each program campus, so the effort is to some degree self-sustaining. To reduce 
the problem cited by both coaches and teachers that training was not appropriate to all levels of 
teacher experience, district staff should further support the effort by offering separate training 
sessions in Balanced Literacy tailored to different levels of experience with the strategies. 

Lack of materials, especially bilingual materials, was a frequently reported limitation of 
the program. This almost certainly contributed to the fact that Hispanic students passed the 
TAAS reading test at a rate of 77% as compared to the overall district passing rate of 85%. It is 
recommended that, through other funding sources, district staff continue to expand the variety of 
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support materials available for Spanish-speaking students. This is especially important in light 
of the fact that, beginning in 2002-03, campus libraries will not have Title VI funding to 
supplement their collections. 

Collaborative efforts with community agencies spawned a great deal of involvement 
among students and their families in literacy and English as a second language. It is 
recommended that the district continue to support collaborative relationships with community 
organizations, especially with the Austin Learning Academy and KLRU. 

Several challenges of this project stemmed from lack of resources, and that lack was 
exacerbated by the cutback in funding due to failure to spend funds the previous year. It is 
recommended that project managers on similar grants schedule a review of campus expenditures 
and the available budget well before the spending cutoff date. 
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